[DOCS-13696] Add new Experiments landing page#35419
[DOCS-13696] Add new Experiments landing page#35419
Conversation
…menu file, added an index page with some initial content
Preview links (active after the
|
iadjivon
changed the title
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
| @@ -0,0 +1,73 @@ | |||
| --- | |||
There was a problem hiding this comment.
NOTE: This page is not new. It is moved from _index to its own page to allow for the new landing page. I fixed some trailing white spaces here and I have a card to edit this doc as well.
The focus of this PR is the content/en/experiments/_index.md page.
| description: Use Datadog Experiments to measure the causal relationship that new experiences or features have on user outcomes. | ||
| aliases: | ||
| - /product_analytics/experimentation/ | ||
| title: Experiments |
There was a problem hiding this comment.
NOTE: This page ignores the mention of warehouse-native data as those content are not yet ready.
I will open another PR to include that content after the warehouse-native docs are ready and can be linked.
jeff-morgan-dd
left a comment
jeff-morgan-dd
left a comment
There was a problem hiding this comment.
Looks good! Nothing blocking, please review a few comments - particularly the one about order of operations.
| 1. **[Minimum Detectable Effects][3]**: Choose appropriately sized MDEs. | ||
| 1. **[Create a metric][4]** to evaluate your experiment. | ||
| 1. **[Create an experiment][5]** to define your hypothesis and optionally calculate a [sample size][8]. | ||
| 1. **[Create a feature flag][6]** and implement it using the [SDK][9] to assign users to the control and variant groups. A feature flag is required to launch your experiment. |
There was a problem hiding this comment.
If a feature flag is required for an experiment, should users create feature flags prior to creating an experiment?
There was a problem hiding this comment.
Great question. Users can create the feature flag or the experiment first, the UI supports both workflows.
Common use cases:
- Engineer is building a feature before the experiment design is finalized -> create flag first for development, add an experiment when you're ready to launch
- PM is planning an experiment before any development kicks off -> create draft exp, run sample size calculation. Once an engineer is ready to start development, they're create a flag and add it to the experiment
| After selecting your experiment’s metrics, use the optional sample size calculator to determine how small of a change your experiment can reliably detect with your current sample size. | ||
|
|
||
| 1. Select the **Entrypoint Event** of your experiment. This specifies _when_ in the user journey they will be enrolled into the test. | ||
| 1. Click **Run calculation** to see the [Minimum Detectable Effects][3] (MDE) your experiment has on your metrics. The MDE is the smallest difference you can detect between your experiment’s variants. |
There was a problem hiding this comment.
The first sentence here can, I think, be clearer with a bit of fine-tuning. The experiment is driving metrics, sort of, but doesn't really have an effect on metrics per se. It's more around what can be detected with a given sample size, which you capture clearly in the 2nd sentence.
| {{< img src="/product_analytics/experiment/exp_feature_flag.png" alt="Set up an experiment on a Feature Flag." style="width:90%;" >}} | ||
|
|
||
| ## Next steps | ||
| 1. **[Defining metrics][2]**: Define the metrics you want to measure during your experiments. |
There was a problem hiding this comment.
This seems like something they would have done already in the course of prepping and launching an experiment. Do you think Next Steps is the right place for it? Would it work as a Further Reading link?
3 participants
What does this PR do? What is the motivation?
Fixes DOCS-13696
/experiments/. (Source content).Merge instructions
Merge readiness:
Additional notes
This page ignores the mention of Warehouse-Native data as those content are not yet ready.
I will open another PR to include that content after the docs are ready and can be linked.